Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bugfix: use a shared loop between sanic and asyncio components #364

Merged
merged 1 commit into from
Feb 20, 2020
Merged

bugfix: use a shared loop between sanic and asyncio components #364

merged 1 commit into from
Feb 20, 2020

Conversation

edaniszewski
Copy link
Contributor

@edaniszewski edaniszewski commented Feb 20, 2020 •
edited
Loading

This PR:

  • Fixes a bug in which the asyncio.Lock for cache operations was causing request failures when the lock was being used because sanic (where the requests originate from and where lock await was occurring) was using a different event loop than the lock.
  • Adds additional logging around errors to better capture error state in logs

I managed to reliably reproduce the issue seen in #362 locally before the fix, and was not able to reproduce after this fix.

KylerBurke
KylerBurke approved these changes Feb 20, 2020
View reviewed changes
Copy link
Contributor

@KylerBurke KylerBurke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is some seriously explicit loop handling 👍

synse_server/api/http.py
@@ -365,9 +401,8 @@ def log_request(request, **kwargs):
try:
async for reading in cmd.read_cache(start, end):
await response.write(ujson.dumps(reading) + '\n')
except Exception as e:
logger.error('failure when streaming cached readings')
logger.exception(e)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

MatthewHink
MatthewHink approved these changes Feb 20, 2020
View reviewed changes
Copy link
Contributor

@MatthewHink MatthewHink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is testing a ton better than previously. The thermistors blipped out for a bit, but they did re-appear. I think there is still an issue, but it's substantially better and we should take this PR.

Scan and synse logs:

missing-thermistors-2020-02-20.txt

missing-thermistors-logs-2020-02-20.txt

@MatthewHink
Copy link
Contributor

^ It looks like i2c and snmp blipped out of the scan in this case.

@edaniszewski
Copy link
Contributor Author

good to know! thanks for the logs. I'll merge this in but leave the associated issue open so we can keep exploring it.

@edaniszewski edaniszewski merged commit 7005716 into v3/staging Feb 20, 2020
@edaniszewski edaniszewski deleted the v3/shared-loop branch February 20, 2020 18:14
@edaniszewski edaniszewski mentioned this pull request Feb 20, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KylerBurke KylerBurke KylerBurke approved these changes

@MatthewHink MatthewHink MatthewHink approved these changes

@marcoceppi marcoceppi Awaiting requested review from marcoceppi

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@edaniszewski @MatthewHink @KylerBurke